TC-DWA:Text Clustering with Dual Word-Level Augmentation

The pre-trained language models, e.g., ELMo and BERT, have recently achieved promising performance improvement in a wide range of NLP tasks, because they can output strong contextualized embedded features words. Inspired by their great success, this paper we target at fine-tuning them to effectively handle the text clustering task, i.e., classic fundamental challenge machine learning. Accordingly, propose novel BERT-based method, namely Text Clustering with Dual Word-level Augmentation (TCDWA). To be specific, formulate self-training objective enhance it dual word-level augmentation technique. First, suppose that each contains several most informative words, called anchor supporting full semantics. We use words as augmented data, which are selected ranking norm-based attention weights Second, an expectation form word augmentation, is equivalent generating infinite features, further suggest tractable approximation Taylor expansion for efficient optimization. evaluate effectiveness TCDWA, conduct extensive experiments on benchmark datasets. results demonstrate TCDWA consistently outperforms state-of-the-art baseline methods. Code available: https://github.com/BoCheng-96/TC-DWA.